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The Clarens Web Services Architecture 
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Clarens is a Grid-enabled web service infrastructure implemented to augment the current batch-oriented Grid 
services computing model in the Compound Muon Solenoid (CMS) experiment of the LHC. Clarens servers 
leverage the Apache web server to provided a scalable framework for clients to communicate with services 
using the SOAP and XML-RPC protocols. This framework provides security, session persistent storage, service 
discovery, and call routing to back-end services. As an implementation policy Clarens uses widely implemented 
standards wherever possible instead of inventing new standards. 

This paper describes the basic architecture of Clarens, while a companion paper describes clients and services 
that take advantage of this architecture. More information and documentation is also available at the Clarens 
web page at http://clarens.sourceforge.net 



1. Introduction 

The ascendance of Grid-like Q technologies has 
been all but necessitated by the sheer volume of data 
produced in both science and commerce. In response 
to this increased uptake in High Energy Physics in 
particular, the traditional batch-oriented implementa- 
tions Q LL2} using home-grown protocols have started 
to adapt to an industry-wide move to standardized 
interfaces and protocols 

In this context the CAIGEE 8] project was started 
to develop a specific application of Grid technologies 
to the area of interactive analysis by end-user physi- 
cists. A diagram showing Clarens as the interface be- 
tween distributed clients and Grid services is shown 
in Figure 
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Figure 1: CAIGEE architecture diagram. 



can include anything from an end-user analysis pack- 
age like ROOT [l0(, web browsers or even web-enabled 
PDAs. Other Clarens servers may also act as clients. 

Back-end services planned or implemented include 
data movement, Grid- wide execution planning and 
scheduling, cluster job scheduling as well as metadata 
catalogs. 



2. Infrastructure 

In order to save development time and improve scal- 
ability, The Clarens server is implemented as an ex- 
tension to the Apache l] multi-process web server 
using the mocLpython extension in the Python byte- 
code compiled language. Most CMS sites have these 
components already installed on processing cluster 
head nodes as part of the Redhat 7.3-based OS 
used. Clarens itself is both architecture and platform- 
dependent by virtue of using Python as an implemen- 
tation language. 

The Clarens architecture is depicted in Figure [21 in 
the order that requests are processed by the server. 
Firstly (top), the Apache server receives an HTTP 
POST or GET request from the client, and invokes 
Clarens based on the form of the URL specified by 
the client. Other URLs are handled as usual by the 
server according to its configuration. Secure Sockets 
Layer (SSL) encrypted connections are handled trans- 
parently by the Apache server, with no special coding 
needed in Clarens itself to decrypt (encrypt) requests 
(responses) . Encryption of network traffic is optional, 
however, in cases where it is not required, without 
exposing client or server credentials. 

After the request has been processed, a response is 
sent back to the client, which is usually encoded as an 
RPC response, but may also be in the form of binary 
data. GET requests returns a file or an XML-encoded 
error message to the client, while XML-RPC or 
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sion of the client's private key can discover the server 
session ID, and also that the server is in possession of a 
private key matching the certificate sent as (1) above. 
In subsequent requests the client should set the client 
session ID as username, and server session ID as pass- 
word in the HTTP basic authentication header. 

Once this certificate and session ID exchange is 
completed, both the client and server certificates can 
be verified against the publicly available CA certificate 
chain, knowing that the other party is in possession 
of a matching private key. 



4. Authorization 



Figure 2: Clarens architecture diagram. 

3. Authentication 

In keeping with the Grid tradition, Clarens uses 
a so-called Public Key Infrastructure (PKI) based 
authentication system that relies on X509 format- 
ted certificates issued by a Certification Authority 
(CA) along with asymmetric encryption using public 
and private keys. The authentication protocol of the 
server is implemented at the application level, thus 
eliminating the need for a custom security layer on 
the client side. 

If an SSL-encrypted connection is used the client's 
certificate is provided to the server as part of the con- 
nection negotiation stage. The Apache server passes 
this information to the Clarens layer. This is the de- 
fault for browser-based clients which in general have 
a well developed client-side PKI security infrastruc- 
ture. In this case the authentication step in initi- 
ated by calling the RPC method system. auth2() 
with no arguments. A user session ID is requested 
by the client by setting the clarens_username cookie 
value in the message header to the requested session 
ID, and setting the clarens_password cookie value to 
BROWSER. Clarens responds by returning its own cer- 
tificate as well as it's part of the session ID encoded as 
an RPC response. In subsequent requests the client 
must set the clarens_password to this server session 
ID. Browsers will automatically send these cookie val- 
ues in subsequent requests. 

In the case of an unencrypted connection, or a client 
not able to send it's certificate as part of the connec- 
tion negotiation phase, the session ID and client cer- 
tificate must be sent using as the username and pass- 
word in the the HTTP basic authentication header 
invoking the system. authO method. The server re- 
sponds with a list of (1) its certificate, (2) the server 
session ID encrypted using the user's public key, and 
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Authorization of clients to access server resources 
(invoking methods, accessing files etc.) is done within 
the framework of a hierarchical Virtual Organization 
(VO) with members uniquely identified by their Dis- 
tinguished Names (DNs) issued by the CAs as part of 
all X509 certificates. 

4.1. Virtual Organization 

Each Clarens server instance manages a tree-like 
VO structure, as shown in Figure rooted in a list 
of administrators. This group, named admins, is pop- 
ulated statically from values provided in the server 
configuration file on each server restart. The list of 
group members is cached in a database Q, as is all 
VO information. The admins group is authorized to 
create and delete groups at all levels. 

Each group consists of two lists of DNs for the group 
members and administrators respectively. Group ad- 
ministrators are authorized to add and delete group 
members, as well as groups at lower levels. The 
group structure is hierarchical because group mem- 
bers of higher level groups are automatically members 
of lower level groups in the same branch. 

The example in Figure|3]dcmonstrates the top-level 
groups A, B, and C, with second level groups A. 1, A. 2, 
and A. 3. 

A more concrete example might be to define groups 
CMS, Atlas, LHCb, and Alice, then for CMS, to de- 
fine CMS. USA, CMS.CERN, CMS. UK, CMS. Germany. At 
the third level, one might define CMS.USA.Caltech, 
CMS.USA.UFL, CMS.USA.FNAL. Management for the 
latter three groups may then be delegated to the insti- 
tutes themselves, thereby implementing a distributed 
trust model that has lower maintenance overhead as 
well as being more representative of the real organi- 
zational structure. 

As a further optimization, the hierarchical in- 
formation in the DNs may also be used to define 
membership, so that only the initial significant 
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Figure 3: Clarens virtual organization diagram. 

(C), state/province (ST), locality/city (L), orga- 
nization (O), organizational unit (OU), common 
name (CN), and e-mail address (Email). An ex- 
ample DN issued by the DOE Science Grid CA 
is /O=doesciencegrid. org/OU=People/CN=John 
Smith 12345 for individuals and 

/O=doesciencegrid. org/OU=Services/CN=host 
/www.mysite . edu for servers. To add all 
individuals to a particular group, only 
/O=doesciencegrid. org/DU=People need to speci- 
fied as a member DN. 

4.2. Access Control Lists 

Execution of RPCs (web service methods) as well as 
mapping of certificate DNs to users on the server sys- 
tem is controlled by a set of hierarchical access control 
lists (ACLs) in a similar fashion to the VO structure 
described above, and modeled after the access control 
(.htaccess) files used by Apache. 

Methods have a natural hierarchical structure 
modeled after Python's module infrastructure. In 
fact all Clarens modules are also Python modules. 
Clarens places no arbitrary restrictions on the depth 
of this hierarchy, but a depth of two or three 
levels is most common, e.g. module .method or 
module . submodule .method. 

An ACL consist of an evaluation order specification 
(allow, deny or deny, allow) followed by a list of DNs 
allowed, groups allowed, DNs denied and groups de- 
nied access. A DN or group granted access to a higher 
level method automatically has access to a lower level 
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Table I Method ACL example. 



Object 


Field 


Value 


mod 


order 


deny, allow 




allow DNs 


/O=doesg . org/QU=People/CN= John Smith 






/Q=doesg . org/QU=People/CN=Ng Siong 




allow groups 


CMS. USA 






CMS . CERN 




deny DNs 


/0=olduni/0U=physics/CN=01d Account 




deny groups 


crackers 


mod .meth 


order 


deny, allow 




allow DNs 






allow groups 


CMS.USA.Caltech 






CMS.USA.UFL 




deny DNs 


/0=Caltech/OU=CACR/CN=Ed Peng 




deny groups 





lower level. The ACL specification is therefore evalu- 
ated from the lowest applicable level to the highest. 

According to the example ACLs in Table [I] when 
the method mod. meth is invoked, the second ACL is 
applied, but when any other method in module mod is 
invoked, the first ACL is applied. 



4.3. User Mapping 



In the traditional batch-oriented Grid architecture, 
being able to start long-running CPU intensive jobs 
as a certain system user is of crucial importance. E.g. 
the Globus toolkit 5] implements the concept of a 
so-called gridmap file that maps system users to cer- 
tificate DNs. It is implemented as a fiat text file with 
two values per line, namely the DN and the system 
user. 

Clarens similarly contains the notion of mapping 
DNs to user names. Instead of a flat file, a structure 
similar to the ACLs described above is used, with the 
username taking the place of the method name, with 
one exception: the deny group and deny individual 
fields are not used, since denying access to process 
creation methods can be done using the method ACLs 
themselves. 

At this point it should be clear that in both the 
VO, ACL, and user mappings specification as little 
information as possible is stored in order to minimize 
search times for list memberships. Searching these 
lists are in the critical path for the invocation of any 
method, though, and must be optimize as far as prac- 
tical. This is done by storing lists of DNs as strings 
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from mod_python import apache 
from clarens_util import 

build_response , write_response 

def echo (req,method_name , args) : 

"""Returns the method argument""" 
response=build_response(req, "echo" ,args) 
write_response (req, response) 
return apache . OK 

methods_list={'echo' :echo}- 
methods_sig={ ' echo ' : [' string, string'] ~} 



Figure 4: Clarens module example. 



4.4. Auditing 



All method invocations are logged with a time- 
stamp in the Apache server log files to provide a record 
of client /server transactions. 



5. Modular Architecture 



Clarens provides a framework for extending its func- 
tionality via new modules installed in subdirectories. 
Each subdirectory would appear as a new method, 
root, e.g. a system subdirectory would have its meth- 
ods accessible as system, method etc. Modules can be 
implemented as either interpreted Python bytecode, 
or as compiled 0/0++ shared libraries if code execu- 
tion speed is important, and are loaded on demand 
by the Python interpreter in each Apache process, 
thereby providing crash-protection between different 
processes, a major consideration for highly available 
servers. 

Provision is also made for users on the server system 
to install their own modules in subdirectories under 
their home directories. These modules are accessible 
using the format ~user .module .method. 

Figure 01 shows an example of the simplest module 
implementation for a method echo, that when placed 
in a file named echo/_init_.py returns its argument 
unchanged when called as echo . echo (argument) . 

Two important elements identifies this as a Clarens 
module: the methods_list and methods_sig struc- 
tures identifying the list of methods and their signa- 
tures. Work is underway to use WSDL as method 
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6. Persistency 

Since the HTTP protocol does not require 1 persis- 
tent connections, it is important that session informa- 
tion be stored persistently on the server side. This 
has the positive side-effect of allowing clients to sur- 
vive server failures or restarts transparently without 
having to re-authenticate themselves to the server in 
those cases. 

The most important session information that is 
stored by the Clarens server is the authentication in- 
formation for each session. The Berkeley database Q 
is used for this purpose. 

7. Scalability, failover, clustering 

Since Clarens is built upon commodity software 
components and standard operating system services, 
it relies on these components to be set up to achieve 
these goals. Specifically, the Apache web, server, the 
filesystem, database and network components needs 
to be configured by the system administrator. 

8. Future developments 

Work is underway to extend Clarens from being an 
essentially client/server system to being a truly dis- 
tributed system in a network of mutually aware peers 
and superpeers that provide services. The most press- 
ing need for large scale Physics analysis is the need for 
truly distributed data catalogs for a variety of Physics 
data and metadata, and of course a matching search 
capability. 

A switch to a relational database for persistent data 
storage is planned to support more advanced data 
management than the Berkeley database's key/value 
mechanism can provide. Along with this change, file 
ACLs will also be implemented. 

Werever practical Clarens aims to be compatible 
with the OGSA framework, with support for SOAP 
and WSDL being important first steps in that direc- 
tion. 



9. Conclusion 

Clarens is a powerful, yet simple web services ar- 
chitecture with a strong emphasis on security for dis- 
tributed virtual organizations. It draws upon a rich 
base of commodity protocols and software components 



In the HTTP 1.1 standard, persistent connections are the 
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to provide a platform for the deployment of analysis- 
oriented web services. 

A companion paper describes services and clients 
that take advantage of this platform 0] . 
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Appendix: Ternary trees 

As pointed out in Section 0] several lists must be 
searched for every method invocation to enforce access 
control, establish group membership and map certifi- 
cate DNs to system user names. 




Figure 5: Ternary tree graph for the words: Clarens 
ternary tree example May 2003. 

The keys for these lists (method, group, or user 
names) are kept in Python dictionary objects, which 
are implemented as hash tables. Although it is cer- 
tainly possible to store the aforementioned lists in 
hash tables, these structures lack an important fea- 
ture, namely the ability to search for initial sub- 
strings. E.g. if we are presented with the query DN 
/0=myorg/OU=People/CN=John Smith, but only the 
string /0=myorg/OU=People is stored. 

Initial substring searches are particularly suited to 
a data structure called a ternary tree |3J that contains 
nodes for less than, equals and greater than compar- 
isons for string fragments. In contrast to the refer- 
ence implementation, the Clarens ternary trees con- 
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When presented with the above DN, the tree can 
be traversed by comparing and branching every two 
characters, and returning the number of characters 
matched if a leaf node is reached, or a failure can be 
signaled if a leaf node is not reached. The returned 
number can be compared with the length of the query 
DN: an exact string match corresponds to an equal 
number of matched characters, while a lower number 



corresponds to a substring match. 

The ternary tree structure is implemented as a 
Python extension module in C. Informal tests with 
10,000 DNs on a 933 MHz PC produces roughly 
300,000 searches/second for the worst case where each 
DN is present in the tree, i.e. the maximum number 
of branches taken per search. 



